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Abstract 



The two-parameter Macdonald polynomials are a central object of algebraic com- 
t^ binatorics and representation theory. We give a Markov chain on partitions of k 

^^ with eigenfunctions the coefficients of the Macdonald polynomials when expanded in 

I— —I the power sum polynomials. The Markov chain has stationary distribution a new 

P^ two-parameter family of measures on partitions, the inverse of the Macdonald weight 

Q^ (rescaled). The uniform distribution on permutations and the Ewens sampling formula 

(-H are special cases. The Markov chain is a version of the auxiliary variables algorithm of 

statistical physics. Properties of the Macdonald polynomials allow a sharp analysis of 
the running time. In natural cases, a bounded number of steps suffice for arbitrarily 
large k. 
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• 1 Introduction 

l> 

f-->) The Macdonald polynomials Px{x;q,t) are a widely studied family of symmetric polynomials 

y—i in variables X = (xi, X2, • • • , x„). Let A^ denote the vector space of homogeneous symmetric 

J> polynomials of degree k (with coefficients in Q). The Macdonald inner product is determined 

IT^ by setting the inner product between power sum symmetric functions px as 

d {px,Pt^) = Sxfj.zx{q,t), 

with 



(1 ^\i \ 
l_^x, ) and ^A = n 



i'^'ai], 



for A a partition of k with Oj parts of size i. 

For each q,t, as A ranges over partitions of k, the Px{x;q,t) are an orthogonal basis 
for A^. Special values of q,t give classical bases such as Schur functions {q = t), Hall- 
Littlewood functions (t = 0) , and the Jack symmetric functions (limit as t — )■ 1 with g° = t). 
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An enormous amount of combinatorics, group theory, and algebraic geometry is coded into 



these polynomials. A more careful description and literature review is in Section 2 



The original definition of Macdonald constructs P\{x;q,t) as the eigenfunctions of a 
somewhat mysterious family of operators Dq^t{z)- This is used to develop their basic prop- 
erties in [39j . A main result of the present paper is that the Macdonald polynomials can be 
understood through a natural Markov chain M(A, A') on the partitions of k. For q,t > 1, 
this Markov chain has stationary distribution 



(1.2) 



7rg,t(A) 



Z 



zx{q,t) 



with Z 



iq,q)k 

(t,q)k 



k-1 



{x,y)k = n (1-2;?/') 



Here zx{q, t) is the Macdonald weight ( 1.1 ) and Z is a normalizing constant. The coefficients 
of the Macdonald polynomials expanded in the power sums give the eigenvectors of M, and 
there is a simple formula for the eigenvalues. 

Here is a brief description of M. From a current partition A, choose some parts to delete: 
call these Xj. This leaves Xjc = X\Xj. The choice of Ajc given A is made with probability 



(1.3) 



wx{Xjc) 



k 



It is shown in Section 2.4 that for each A, wx{-) is a probability distribution with a simple- 
to- implement interpretation. Having chosen Ajc, choose a partition fi of size |A| — \Xjc\ with 
probability 



(1.4) 



7roo,i(Ai) 



T^n 



1 



ai(^) 



Adding /i to Ajc gives a final partition z/. These two steps define the Markov chain M{X, v) 
with stationary distribution -Kq^f It will be shown to be a natural extension of basic al- 
gorithms of statistical physics: the Swendsen-Wang and auxiliary variables algorithms. 
Properties of the Macdonald polynomials give a sharp analysis of the running time for M. 
Section 2 gives background on Macdonald polynomials (Section 2.1), Markov chains 



(Section 2.2), and auxiliary variables algorithms (Section 2.3). The Markov chain M is 
shown to be a special case of auxiliary variables and hence is reversible with 7rg^((A) as 



stationary distribution. Section 2.4| reviews some of the many different measures used 
on partitions, showing that w\ and tToo,* above have simple interpretations and efficient 
sampling algorithms. Section 2.4 also presents simulations of the measure i^q^tiX) using M. 



This gives an understanding of TTq^t] it also illustrates (numerically) that a few steps of M 
suffice for large k while classical sampling algorithms (rejection sampling or Metropolis) 
become impractical. 

The main theorems are in [Section 31 The Markov chain M is identified as one term of 
Macdonald operators Dq^t{z)- The coefficients of the Macdonald polynomials in the power 
sum basis (suitably scaled) are shown to be the eigenfunctions of M with a simple formula 
for the eigenvalues. Needed values of the eigenvectors are derived. A heuristic overview of 



the argument is given (Section 3.2), which may be read now for further motivation. 



The main theorem is an extension of earlier work by Hanlon |15[ [3T] giving a similar 
interpretation of the coefficients of the family of Jack symmetric functions as eigenfunctions 



of a natural Markov chain: the MetropoHs algorithm on the symmetric group for generating 



from the Ewens sampling formula. Section 4 develops the connection to the present study. 



Section 5 gives an analysis of the convergence of iterates of M to the stationary dis- 
tibution -Kq^t for natural values of q and t. Starting from (A;), it is shown that a bounded 
number of steps suffice for arbitrary k. Starting from 1^ , order logfc steps are necessary 
and sufficient for convergence. 

2 Background and examples 

This section contains needed background on four topics: Macdonald polynomials, Markov 
chains, auxiliary variables algorithms, and measures on partitions and permutations. Each 
of these has a large literature. We give basic definitions, needed formulae, and pointers to 



literature. Section 2.3 shows that the Markov chain M of the introduction is a special case 



of the auxiliary variables algorithm. [Section 2.4 shows that the steps of the algorithm are 



easy to run, and has numerical examples. 

2.1 Macdonald polynomials 

Let A„ be the algebra of symmetric polynomials in n variables (coefficients in Q). There 
are many useful bases of A„; the monomial {m\}^ power sum {px}-, elementary {ca}, homo- 
geneous {/ia}j and Schur functions {s\} are bases whose change of basis formulae contain 
a lot of basic combinatorics [50, Chap. 7], [39l Chap. I]. More esoteric bases such as the 
Hall-Littlewood functions {H\{q)}, zonal polynomials {Zx}, and Jack symmetric functions 
{J\{o)} occur as the spherical functions of natural homogeneous spaces [39]. In all cases, 
as A runs over partitions of A;, the associated polynomials form a basis of the vector space 
A^: homogeneous symmetric polynomials of degree k. 

Macdonald introduced a two-parameter family of bases P\{x; q, t) which, specializing g, t 
in various ways, gives essentially all the previous bases. The Macdonald polynomials can 
be succinctly characterized by using the inner product {p\,Pfj) = Sx^z\(q,t) with zx{q,t) 



from (1.1). This is positive definite [39^ VI (4.7)] and there is a unique family of symmetric 
functions P{x; q, t) such that (Pa, P^l)q,t = if A / /x and Pa = Z]^<a ux^j.m^ with uaa = 1 
[59t VI (4.7)]. The properties of Pa are developed by studying Pa as the eigenfunctions of 
a family of operators Dq^t{z) from A„ to A„. 

Define an operator Tu^x, on polynomials by Tu,xJ{xi, • • . , x„) = f{xi, ... , uxi, . . . , rc„). 
Define Dq^t{z) and Dq^ by 

n ^ n 

(2.1) Dq,{z) = ^ Dl,z^ = - E d«*(^) ^"' n (l + ^^^"'^^^^ 



r=0 ° weSn j=l 



q,Xi I , 



where 5 = (n — 1, n — 2, . . . , 0), a^ is the Vandermonde determinant and x'^ = x\^ ■ ■ ■ xZ" for 
7 = (7i> ■■■, In)- For any r = 0, 1, . . . , n, 

(2.2) Dl, = Y,Mx;t)llTq,x,, 

I iei 



Ml VI (3.4),]. 



where the sum is over all r-element subsets / of {1, 2, . . . , n} and 

(2.3) Mix, t) = - (n Tt,.) as = r(-i)/2 n ^^i^^ 

as V-"--"- / J^J- Xi - Xj 

Macdonald |39^ VI (4.15)] shows that the Macdonald polynomials are eigenfunctions of 



(2.4) 



D,,t{z)Py,{x- q,t) = J{{l + zq'^H^-') Px{x; q, t). 



i=l 



This implies that the operators D^^ commute, and have the P\ as eigenfunctions with 
eigenvalues the rth elementary symmetric function in {g'^'t""*}. We will use D^^ in our 
work below. The D^^ are self-adjoint in the Macdonald inner product {D^^f, g) = (/, D^^g). 
This will translate into having TTq^t as stationary distribution. 

The Macdonald polynomials may be expanded in the power sums [391 VI (8.19)], 



(2.5) 



Px{x;q,t) 



cx{q,t', 



E 



-'llil-t^^)X^iq,t) 



PfA 



with [39, VI (8.1)] c\{q,t) = Yl^fzxi^ ~ g^^^H'^'^^"'"^) where the product is over the boxes in 
the shape of A, a{s) the arm length and l{s) the leg length of box s. The Xp{q, t) are closely 
related to the two-parameter Kostka numbers Kn\{q,t) via [391 VI (8.20)], 



(2.6) 






0, K^x{q,t) = }_^z-'x';>X^{q,t) 



\^^-i. 



Section 3 



with Xp the characters of the symmetric group for the Ath representation at the pth conju- 
gacy class. These K^x{q,t) have been a central o 
IS], [23], [26], [271 Eg EH], [30]. The main result of 
are the eigenfunctions of the Markov chain M. 

The Macdonald polynomials used here are associated to the root system An- Macdonald 
[40| has defined analogous functions for the other root systems using similar operators. In 



3Ject of study in algebraic combinatorics 



shows that X^{q, t) r].(l - qi" 



a major step forward, Cherednik [T3] gives an independent development in all types, using 
the double affine Hecke algebra. See j39^ Hi] for a comprehensive treatment. Using this 



language. Ram and Yip [48] give a "formula" for the Macdonald polynomials in general 
type. In general type the double affine Hecke is a powerful tool for understanding actions. 
We believe that our Markov chain can be developed in general type if a suitable analogue 
of the power sum basis is established. 



2.2 Markov chains 

Let A:" be a finite set. A Markov chain on X may be specified by a matrix M{x, y) > 
0, "^yM^x,!/) = 1. The interpretation being that M{x,y) is the chance of moving from 
X to 2/ in one step. Then M'^{x,y) = '^^M{x,z)M{z,y) is the chance of moving from x 
to y in two steps, and M {x,y) is the chance of moving from x to y in ^ steps. Under 
mild conditions, always met in our examples, there is a unique stationary distribution 



7r(x) > 0, Ylx'^i^) ~ ^- This satisfies Ylx'^(^)^i^^y) ~ '^iv)- Hence, the (row) vector vr 
is a left eigenvector of M with eigenvalue 1. Probabilistically, picking x from vr and taking 
one further step in the chain leads to the chance 7r(y) of being at y. 

All of the Markov chains used here are reversible, satisfying the detailed balance con- 
dition 7r(x)M(x,y) = 7r(y)M(y,x), for all x,y in X. Set L^(Af) to be {/ : Af -^ M} 
with (/i,/2) = Z^x ^(2;)/i(a;)/2(2;). Then M acts as a contraction on L'^{X) by Mf{x) = 
"^ M{x,y)f{y). Reversibility is equivalent to M being self-adjoint. In this case, there is 
an orthogonal basis of (right) eigenfunctions /, and real eigenvalues f3i, 1 = /3o > /3i > 
• • • > (^\x\-i ^ ~1 with Mfi = Pifi. For reversible chains, if fi{x) is a left eigenvector, then 
fi{x)/Tr{x) is a right eigenvector with the same eigenvalue. 

A basic theorem of Markov chain theory shows that M^{y) = M [x, y) — )• 7r(y). (Again, 

oo 

there are mild conditions, met in our examples.) The distance to stationarity can be mea- 
sured in L^ by the total variation distance: 



(2.7) 



Mi 



vr 



TV 



max 
A<zx 



M' 



x,A)-7:{A) =lY,W{x,y)-7:{y) 



Distance is measured in L^ by the chi-squared distance: 



(2.^ 



Mi 



vr 



E 



{M\x,y)-Tr{y)y 
7r(y) 



\x\-i 



21 



where /, is the eigenvector /j, normalized to have L^-norm 1. 
equality shows 



The Cauchy-Schwarz in- 



(2.9) 



M^-TT 



< 



TV 



Mi 



vr 



Using these bounds calls for getting one's hands on eigenvalues and eigenvectors. This can 
be hard work, but has been done in many cases. A central question is this: given M, e > 0, 
and a starting state x, how large must (. be so that ||M| — vtHtv < e? 

Background on the quantitative study of rates of convergence of Markov chains is treated 
in the textbook of Bremaud [11]. The identities and inequalities that appear above are 
derived in the very useful treatment by Saloff-Coste |49| . He shows how tools of analysis 



can be brought to bear. The recent monograph of Levin, Peres and Wilmer |37j is readable 
by non-specialists and covers both analytic and probabilistic techniques. 



2.3 Auxiliary variables 

This is a method of constructing a reversible Markov chain with vr as stationary distribution. 



It was invented by Edwards and Sokal |20j as an abstraction of the remarkable Swendsen- 
Wang algorithm. The Swendsen-Wang algorithm was introduced as a superfast method 
for simulating from the Ising and Potts models of statistical mechanics. It is a block-spin 
procedure which changes large pieces of the current state. A good overview of such block 
spin algorithms is in [32] . The abstraction to auxiliary variables is itself equivalent to several 
other classes of widely used procedures, data augmentation and the hit-and-run algorithm. 
For these connections and much further literature, see [li 



To describe auxiliary variables, let 7r(x) > 0, Ylx ^(^) = 1 be a probability distribution 
on a finite set X Let / be an auxiliary index set. For each x & X, let Wx{i) be a probability 
distribution on I (the chance of moving to i). These define a joint distribution f{x,i) = 
7r{x)wx{i) and a marginal distribution m{i) = X^2./(a;,i). Let f{x\i) = f{x,i)/m{i) denote 
the conditional distribution. The final ingredient needed is a Markov matrix Mi(x,y) with 
f{x\i) as reversing measure {f{x\i)Mi{x,y) = f{y\i)Mi{y,x) for all x,y). This allows for 
defining 

(2.10) M{x, y) = Y^ Wx{i)M,{x, y). 

i 

The Markov chain M has the following interpretation: from x, choose i € I from Wx{i) and 
then y £ X from Mi{x,y). The resulting kernel is reversible with respect to vr: 

7r{x)M{x,y) ='^7:{x)wx{i)Mi{x,y) = ^T^{y)wy{i)Mi{y,x) 

i i 

= 7i"(y) ^ Wy{i)Mi{y, x) = TT{y)M{y, x). 

i 

We now specialize things to Vk, the space of partitions of k. Take X = Vk, I = U^^^Vi. 



The stationary distribution is as in (1.2): 
(2.11) 7r(A) = 7r,,i(A) 



From X £ Vk, the algorithm chooses some parts to delete, call these Xj, leaving parts 
Xjc = X\Xj. Thus if A = 322111 and Xj = 31, Xjc = 2211. We allow Aj = A but demand 
Xj 7^ 0. Clearly, A and Xj determine Xjc and (A, Ajc) determine Xj. We let Xjc be the 
auxiliary variable. The choice of Xjc given A is made with probability 

k / 



-(>.^)-^n (:£!,)''■-" 



(2.12) ';■ 



(^i\X) \ I i ^\ai{X)-ai{\jc) 

\n 

i=l 



bn :£!,('•-') 



Thus, for A = 1^232; Aj = 13, Ajc = 1^23; wxiXjc)='—^Q{l){l){q-l){q^-l)%q^-l). 



It is shown in Section 2.4 below that wx{Xjc) is a probability distribution with a simple 



interpretation. Having chosen Xj with < |Aj| < k, the algorithm chooses fj, h |Aj| with 



probability iToo,t{fJ') given in (1.4). Adding these parts to Xjc gives v. More carefully, 

(2.13) Ma,. (A, v) = TT^^tifi) = ^3Y- n ( 1 - ^ ) 

Here it is assumed that Xjc is a part of both A and v; the kernel Mxj^ (A, i') is zero otherwise. 



sample from. The joint density /(A, Xjc) = 7r{X)wx{Xjc) is proportional to /(A|Ajc) and to 
^ ■ ' n.^"'(')K(A)-a,(A,c))!- 



The normalizing constant depends on Xjc but this is fixed in the following. We must now 
check reversibility of /(A|Ajc), Mxjd^,!^)- For this, compute f{X\Xjc)Mxjc{y^,i^) (up to a 
constant depending on Xjc) as 

n •(! — l/f )ai(A)+a,(j/) 



Yl.ia.{x)+a,{u) (a.(A) - a,(Ajc))! {a,{u) - ai{Xjc))l ' 
This is symmetric in A, i^ and so equals f{i'\Xjc)M\j^{i', A). This proves the following: 
Proposition 2.1. With definitions (|2.11[)-([2.14), the kernel onVk, 



M{X,v) = Y,wx{Xjc)Mx,,M,v) 

generates a reversible Markov chain with TTg^tW o-s stationary distribution. 
Example 1. With A; = 2, let 

z it' - 1) , z (t-iV (i-g)(i-a 

From the definitions, with rows and columns labeled (2), 1^, the transition matrix is 
(2.15) 



M 



2 V t 2 V t 



q-^lf l\ 4t + (g-l)(t-l) 
\g+12V t) 2{q+l)t J 



t+l t-l 

2t I {q-l){t + l) At+{q-l){t-l) 



+ 1 g + 1 



In this k = 2 example, it is straightforward to check that iTq^t sums to 1, the rows of M sum 
to 1, and that TTq^t{>')M{X,u) = 7rq,i(i/)M(z^, A). 

2.4 Measures on partitions and permutations 



The measure TTq^t of ( 1.2 ) has familiar specializations: to the distribution of conjugacy classes 
of a uniform permutation {q = t), and the Ewens sampling measure (g" = t — ;• 0). After 
recalling these, the measures wxj^ (•) and Mxjc (A, •) used in the auxiliary variables algorithm 
are treated. Finally, there is a brief review of the many other, nonuniform distributions used 
on partitions Vk and permutations Sk ■ Along the way, many results on the "shape" of a 
typical partition drawn from TTg^t appear. 

2.4.1 Uniform permutations {q = t) 

If o" is chosen uniformly on S^, the chance that the cycle type of cr is A is 1/zx = TTq^q(X). 
There is a healthy literature on the structure of random permutations (number of fixed 
points, cycles of length i, number of cycles, longest and shortest cycles, order, . . . ). This is 
reviewed in ^22t i46j . which also contain extensions to the distribution of conjugacy classes 
of finite groups of Lie type. 

One natural appearance of the measure l/zx comes from the coagulation/fragmentation 
process. This is a Markov chain on partitions of k introduced by chemists and physicists 

7 



to study clump sizes. Two parts are chosen with probabihty proportional to their size. If 
different parts are chosen, they are combined. If the same part is chosen twice, it is split 
uniformly into two parts. This Markov chain has stationary distribution Xj z\. See [2| for 
a review of a surprisingly large literature and jl7j for recent developments. These authors 
note that the coagulation/fragmentation process is the random transpositions walk, viewed 



on conjugacy classes. Using the Metropolis algorithm (as in Section 2.4.6 below) gives a 
similar process with stationary distribution TTg^f. 

Algorithmically, a fast way to pick A with probability 1/za is by uniform stick-breaking: 
Pick Ui G {1, . . . ,k} uniformly. Pick U2 G {1, . . . ,k — Ui} uniformly. Continue until the 
first time T that the uniform choice equals its maximum value. The partition with parts 
Ui, U2, . . . ,Ut equals A with probability l/zx- 

2.4.2 Ew^ens and Jack measures 

Set q = t" and let t — )• 1. Then vrg^i(A) converges to 

Z oi^W 
(2.16) 7r„(A) = — a~^(^), Z = -^— '■ , £(A) the number of parts of A. 

z\ nj=i (^a + 1) 

In population genetics, setting a = 1/0, with 6 > a, "fitness parameter," this measure 
is called the Ewens sampling formula. It has myriad practical appearances through its 
connection with Kingman's coalescent process, and has generated a large enumerative liter- 
ature in the combinatorics and probability community [U [32| ITT] . It also makes numerous 
appearances in the statistics literature through its occurrence in non-parametric Bayesian 
statistics via Dirichlet random measures and the Dubins-Pitman Chinese restaurant process 
124], 147, sec. 3.1]. 

Algorithmically, a fast way to pick A with probability vrx/g(A) is by the Chinese restau- 
rant construction. Picture a collection of circular tables. Person 1 sits at the first table. 
Successive people sit sequentially, by choosing to sit to the right of a (uniformly chosen) 
previously seated person (probability 9) or at a new table (probability 1 — 6). When k peo- 
ple have been seated, this generates the cycles of a random permutation with probability 
TTi/Q. It would be nice to have a similar construction for the measures TTg^f 

The Macdonald polynomials associated to this weight function are called the Jack sym- 
metric functions f39, VI Sect. 1]. Hanlon [151 EI] uses properties of Jack polynomials to 



diagonalize a related Markov chain; see Section 4 When a = 1/2, the Jack polynomials 



become the zonal-spherical functions of GLn/On. Here, an analysis closely related to the 
present paper is carried out for a natural Markov chain on perfect matchings and phyloge- 
netic trees [HI Chap. X], ll6j. 

2.4.3 The measure wx 

Fix \\- k with £ parts and q > 1. Define, for J C {1, . . . , ^}, J / 0, 

(2.17) -A(j)=^n('^'"-i)- 

The aiixiliary variables algorithm for sampling from TTg^t involves sampling from w\{J), and 



setting \j = {Xi : i G J} (see (1.3) and (2.12)). The measure w\{J) has the following 



interpretation, which leads to a useful sampling algorithm: Consider k places divided into 
blocks of length Aj : 

Xi + --- + Xi = k. 



Ai A2 A; 

Flip a 1/q coin for each place. Let, for 1 < i < i, 

I 1 if the ith block is not all ones, 
1 otherwise. 

Thus P(X, = 1) = 1 - l/q^\ Let J = {i : Xi = 1}. So P{J = 0} = 1 - 1/q'' and 

(2.19) p{j\j / 0} = T^ n (1 - i) n i = -a(^). 

This makes it clear that summing wx{J) over all non-empty subsets of {1, ... ,i} gives 1. 

The simple rejection algorithm for sampling from w\ is: Flip coins as above. If J 7^ 0, 
output Xj = {Xi : i G J}. If J = 0, sample again. The chance of success is 1 — l/q^. Thus, 
unless q is very close to 1, this is an efficient algorithm. 

As q tends to infinity, wx converges to point mass at J = {1, . . . , /c}. As q tends to one, 
w\ converges to the measure putting mass Xi/k on {i}. 



2.4.4 The measure vr, 



00, i 



Generating from the kernel Mxjc{X,iy) of (2.13) with r = |A\Ajc|, requires generating a 
partition in Vr from 

ai{fi) 



w.)^(^)in(>-i)' 



This measure has the following interpretation: Pick fi^^' h r with probability 1/z (i) . This 
may be done by picking a random permutation i n Sr uniformly and reporting the cycle 

above. For each part fi- 



Section 2.4.1 



decomposition, or by the uniform stick-breaking of 
of fi^^' , flip a 1/t coin fi- times. If this comes up tails at least once, and this happens 
simultaneously for each i, set /u = /x'^-*. If some part of /u^^^ produces all heads, start again 
and choose fi^"^' h r with probability 1/z (2) .... The chance of failure is 1/t, independent 
of r. Thus, unless t is close to 1, this gives a simple, useful algorithm. 

The shape of a typical pick from iToo.t is described in the following section. When t tends 
to infinity, the measure converges to 1/z^. When t tends to one, the measure converges to 
point mass at the one part partition (r). 

2.4.5 Multiplicative measures 

For T] = {r]i,r]2, . . ■ , rji^), rji > 0, define a probability on V^ (equivalently, S^) by 

k 



(2.20) -.(A) = fn^r^'^ with Z'^ = Y^^llrj 



ai{fi) 



«=1 fihk ^ 



Such multiplicative measures are classical objects of study. They are considered in |1] 
and [52j, where many useful cases are given. The measures TTg^t fall into this class with 

rji = I ill) - If ^ = (2^1 ; 2:2, . • • ) and y = {yi,y2, ■ ■ ■) are two sequences of numbers and 
Vx{X) is a multiplicative basis of A^ such as {ex}, {px},{hx}, setting i]i = Vi{x)Vi{y) 
gives vr^(A) = —Vx{x)Vx{y)- This is in rough analogy to the Schur measures defined in 
Section 2.4.7 For the choices ex, Px, hx, with Xi, yj positive numbers, the associated 
measures are positive. The power sums, with all Xi = a, yi = b, gives the Ewens measure 
with a = ah. Setting xi = yi = c, Xi = yj = otherwise, gives the measure l/zx after 
normalization. To our knowledge, general multiplicative measures have not been previously 
studied. Multiplicative systems are studied in [39, VI Sect. 1 Ex.]. 



It is natural to try out the simple rejection algorithms of Section 2.4.3 and Section 2.4.4 
for the measures tTj,. To begin, suppose that < r/j < 1 for all i. The measure tt,, has the 
following interpretation: Pick A' G Vk with probability l/zy- As above, for each part of A' 
of size i, generate a random variable taking values 1 or with probability r]i, 1 — r]i. If the 
values for all parts equal 1, set A = A'. If not, try again. For more general rji, divide all r]i 
by V* — maxr/j, and generate from rji/ijl. This yields the measure tTj, on partitions. 

Alas, this algorithm performs poorly for ry, and k in ranges of interest. For example, 
with rji = ^j^ for i = 2, g = 4, when k = 10, 11, 12, 13, the chance of success (empirically) 



is 1/2000, 1/4000, 1/7000, 1/12000. We never succeeded in generating a partition for any 
k> 15. 

The asymptotic distribution of the parts of a partition chosen from tTtj when k is large 
can be studied by classical tools of combinatorial enumeration. For fixed values of q, t, these 
problems fall squarely into the domain of the logarithmic combinatorial structures studied 



in [Ij . A series of further results for more general r] have been developed by Jiang and Zhao 
|34j . The following brief survey of their results gives a good picture of typical partitions. 

Of course, the theorems vary with the choice of t/j. One convenient condition, which 
includes the measure Tiq^t for fixed q,t > 1, is 



(2.211 



E 



(li - 1) 



< CXD. 



Theorem 2.2. Suppose r/j, 
(2.20), then, for j large: 



1 < i < 00, satisfy (2.21). If X & Vk is chosen from vTrj of 



For any j, the distribution of (ai(A), . . . , aj{X)) converges to the distribution 

(2.22) of an independent Poisson vector with parameters iji/i, i < i < j. 

The number of parts of A has mean and variance asymptotic to log k 

(2.23) and, normalized by its mean and standard deviation, 

a limiting standard normal distribution. 

The length of the k largest parts of A converge to 

(2.24) the Poisson-Dirichlet distribution [51 12^11^ . 
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These and other resuhs from |H |3l] show that the parts of a random partition are quite 
similar to the cycles of a unformly chosen random permutation, with the smah cycles having 
slightly adjusted parameters. These results are used to give a lower bound on the mixing 



time of the auxiliary variables Markov chain in Proposition 3.2 below 



2.4.6 Simulation results 



Table 1 



Partition A h 10 


Probability TTq^2,t=iW 


Partition A h 10 


Probability tt^^ 


=2,t=4(A) 


10 


0.164003 




4,4,2 


0.018177 




9,1 


0.121365 




4,4,1,1 


0.010098 




8,2 


0.081762 




4,3,3 


0.016955 




8,1,1 


0.045423 




4,3,2,1 


0.030520 




7,3 


0.068948 




4,3,1,1,1 


0.005652 




7,2,1 


0.062054 




4,2,2,2 


0.004120 




7,1,1,1 


0.011491 




4,2,2,1,1 


0.006867 




6,4 


0.063387 




4,2,1,1,1,1 


0.001272 




6,3,1 


0.053214 




4,1,1,1,1,1,1 


0.000047 




6,2,2 


0.021552 




3,3,3,1 


0.004745 




6,2,1,1 


0.023946 




3,3,2,2 


0.005765 




6,1,1,1,1 


0.002217 




3,3,2,1,1 


0.006405 




5,5 


0.030873 




3,3,1,1,1,1 


0.000593 




5,4,1 


0.049942 




3,2,2,2,1 


0.003459 




5,3,2 


0.037734 




3,2,2,1,1,1 


0.001922 




5,3,1,1 


0.020963 




3,2,1,1,1,1,1 


0.000214 




5,2,2,1 


0.016980 




3,1,1,1,1,1,1,1 


0.000006 




5,2,1,1,1 


0.006289 




2,2,2,2,2 


0.000140 




5,1,1,1,1,1 


0.000349 




2,2,2,2,1,1 

2,2,2,1,1,1,1 

2,2,1,1,1,1,1,1 

2,1,1,1,1,1,1,1,1 

1,1,1,1,1,1,1,1,1,1 


0.000389 
0.000144 
0.000016 
0.000001 
0.000000 





The distribution 7rg_t(A) can be far from uniform. An example, with k 
' 0.16, ^4,2(1^) 



Table 1 



vr4,2(10) 



10, q = 4,t = 2, 
0. The auxiliary variables algorithm for 



is shown in 

the measure TTq^t has been programmed by Jiang and Zhao |34j . It seems to work well over 

a wide range of q and t. A tiny example, 100 steps when k = 10, q 

in 



4, t = 2, is shown 

Table 2 A comparison of the simulations with the exact distribution (easily computed 

10) shows perfect agreement. In our experiments, the choice of q and 



from (1.2) when k 



t does not seriously affect the running time, and simulations seem possible for k up to 10^. 
The distribution of the largest part, for q = 4:, t = 2 and k = 10, k = 100, and k = 1000, 



based on 10^ steps of the algorithm, is shown in Figure 1 Comparison with the limiting 



results of Theorem 2.2 above seems good. The blip at the right side of the figures comes 



from (k); the rest of the distribution follows the limit (2.24) approximately. 



We have compared the auxiliary variables algorithm with the rejection algorithm of 



Section 2.4.5 and the Metropolis algorithm. As reported in Section 2.4.5, rejection fails 
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Table 2 



Sample 100-step walk for Auxiliary Variables 



1. 10 


26. 


6,4 


51. 


6,2,1,1 


76. 


7,2,1 


2. 4,3,3 


27. 


10 


52. 


10 


77. 


10 


3. 6,3,1 


28. 


4,3,2,1 


53. 


7,3 


78. 


7,2,1 


4. 5,5 


29. 


8,1,1 


54. 


8,2 


79. 


9,1 


5. 9,1 


30. 


8,2 


55. 


6,2,2 


80. 


5,4,1 


6. 8,1,1 


31. 


7,3 


56. 


6,4 


81. 


10 


7. 6,2,2 


32. 


9,1 


57. 


4,2,211 


82. 


6,3,1 


8. 9,1 


33. 


8,2 


58. 


5,3,2 


83. 


6,3,1 


9. 4,4,2 


34. 


8,2 


59. 


6,4 


84. 


5,4,1 


10. 4,4,1,1 


35. 


8,2 


60. 


10 


85. 


8,1,1 


11. 4,3,1,1,1 


36. 


10 


61. 


9,1 


86. 


5,3,2 


12. 7,2,1 


37. 


7,1,1,1 


62. 


6,3,1 


87. 


5,3,1,1 


13. 5,3,1,1 


38. 


10 


63. 


4,3,3 


88. 


5,2,2,1 


14. 6,4 


39. 


5,3,2 


64. 


10 


89. 


10 


15. 10 


40. 


4,3,3 


65. 


5,5 


90. 


5,3,2 


16. 5,3,2 


41. 


8,2 


66. 


8,2 


91. 


8,2 


17. 4,3,3 


42. 


7,3 


67. 


5,4,1 


92. 


5,3,2 


18. 9,1 


43. 


6,3,1 


68. 


3,3,2,1,1 


93. 


6,3,1 


19. 7,3 


44. 


10 


69. 


6,4 


94. 


5,4,1 


20. 7,3 


45. 


5,5 


70. 


6,1,1,1,1 


95. 


4,3,2,1 


21. 5,3,2 


46. 


6,3,1 


71. 


4,3,2,1 


96. 


7,3 


22. 5,3,1,1 


47. 


8,1,1 


72. 


5,4,1 


97. 


7,2,1 


23. 5,3,1,1 


48. 


6,1,1,1,1 


73. 


10 


98. 


7,2,1 


24. 6,3,1 


49. 


10 


74. 


5,2,1,1,1 


99. 


5,2,2,1 


25. 5,3,2 


50. 


9,1 


75. 


5,2,2,1 


100. 


4,2,2,1,1 



completely for n > 15. The Metropolis algorithm we used works by simulating permu- 
tations from TTq^t hfted to Sk- From the current permutation a, propose a' by making a 
random transposition (all (g) choices equally likely). If iTq^tif^') > '^q,ticr), move to a'. If 
'^q,t{o'')/Trg^t{o') < 1, flip a coin with probability iTq^tW)/'Kq^t{(^) and move to a' if the coin 
comes up heads; else stay at a. For small values of A;, Metropolis is competitive with aux- 
iliary variables. Jiang and Zhao have computed the mixing time for k = 10, 20, 30, 40, 50 
by a clever sampling algorithm. For g = 4, t = 2, the following table shows the number of 
steps required to have total variation distance less than 1/10 starting from the partition 
(A;). Also shown is p{k)^ the number of partitions of k, to give a feeling for the size of the 
state space. 



k 


10 


20 


30 


40 


50 


Aux 


1 


1 


1 


1 


1 


Met 


8 


17 


26 


37 


53 


p{k) 


42 


627 


5604 


37338 


204,226 



The theorems of Section 3 show that auxiliary variables requires a bounded number of 
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Figure 1: Left: Probability of largest part i for partitions oi k — 10 under TTg^t with q = 
Right: Probability of largest part i for partitions of fc = 100 under tt^^j with (7 = 4, t = 2. 



4, t 



steps for arbitrary k. In the computations above, the distance to stationarity after one step 
of the auxiliary variables is 0.093 (within a 1% error in the last decimal) for A; = 10, . . . , 50. 
For larger k (e.g., k = 100), the Metropolis algorithm seemed to need a very large number 
of steps to move at all. This is consistent with other instances of auxiliary variables, such 
as the Swendsen-Wang algorithm for the Ising and Potts model (away from the critical 
temperature; see |9j). 



2.4.7 Other measures on partitions 

This portmanteau section gives pointers to some of the many other measures that have been 
studied on Vk,Sk- Often these studies are fascinating, deep, and extensive. All measures 
studied here seem distinct from iTq^f 

A remarkable two-parameter family of measures on partitions has been introduced by 
Jim Pitman. For > 0, < a < 1, and \\- k with ^ parts, set 



where 



PeA>^) 



Q{a,m) 



k\ 



{ot/-l) 



zx{e + l-a)^ 



M^n[(i-«) 



(i,i-i) 



aj{X) 



i=i 



e{e + a).. 



if m = 0, 
+ {m — l)a) for m = 1, 2, 3, 



These measures specialize to 1/zx (0 = l,a = 0), and the Ewens measure {9 fixed, 
a = 0), see j47l sec. 3.2]. They arise in a host of probability problems connected to stable 
stochastic problems of index a. They are also being used in applied probability connected 
to genetics and Bayesian statistics. They satisfy elegant consistency properties as k varies. 
For example, deleting a random part gives the corresponding measure on Vk-i- For these 
and many other developments, see the book-length treatments of [7], |47t sec. 3.2]. 
One widely studied measure on partitions is the Plancherel measure, 

p{\) = fiXf/kl, 

with /(A) the dimension of the irreducible representation of S^ associated to shape A. This 
measure was perhaps first studied in connection with Ulam's problem on the distribution 
of the length of the longest increasing sequence in a random permutation; see |381 |5T] . For 
extensive developments and references, see [T|l35j. 
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The Schur measures of |10^ H3j SU US] are generalizations of the Plancherel measure. 
Here the chance of A is taken as proportional to sx{x)sx{y), with sx the Schur function 
and x,y collections of real- valued entries. Specializing x and y in various ways yields a 
variety of previously studied measures. One key property, if the partition is "tilted 135°" to 
make a v-shape and the local maxima projected onto the x-axis, the resulting points form 
a determinantal point process with a tractable kernel. This gives a fascinating collection of 
shape theorems for the original partition. 

One final distribution, the uniform distribution on Vk, has also been extensively studied. 
For example, a uniformly chosen partition has order tt/vGA; parts of size 1, the largest part 
is of size (ydk/i:) ■ log(v6fc/7r), the number of parts is of size v6fclog(/c/(27r)). A survey 
with much more refined results is in [21j . 

The above only scratches the surface. The reader is encouraged to look at |43| HH H5] 
to see the breadth and depth of the subject as applied to Gromov-Witten theory, algebraic 
geometry, and physics. The measures there seem closely connected to the "Plancherel dual" 
of our vTg^i. This dual puts mass proportional to c(A)c'(A) on A, with c, c' the arm-leg length 



products defined in Section 3. 1| below. 



3 Main results 

This section shows that the auxiliary variables Markov chain M with stationary distribu- 
tion 7rg^j(A), A E Pfc, is explicitly diagonalizable with eigenfunctions /a(/u) essentially the 
coefficients of the Macdonald polynomials expanded in the power sum basis. The result 



is stated in Section 3.1 The proof, given in Section 3.3, is somewhat computational. An 



explanatory overview is in Section 3.2 In Section 5 these eigenvalue/eigenvector results 



are used to bound rates of convergence of M. 



3.1 Statement of main results 

Fix g, t > 1 and k > 2. Let M{X,fj,) = J2x c ^a(-^j=)-^^Ajc(A, /u) be the auxiliary variables 
Markov chain on Vk- Here, wx{-) and MA^c(A,/i) are defined in (2.12), (2.13), and studied 
in [Section 2.4.3| and [Section 2.4.41 For a partition A, let 

(3.1) cxiq, t) = n (l - Q^'^-'^t'^-'^^') and c'x{q, t) = J] (^ " Q^^'^^'t'^'^) , 

seA sex 

where the product is over the boxes in the shape A, and a{s) is the arm length and l{s) the 
leg length of box s [Ml VI (8.1)]. 

Theorem 3.1. 

(1) The Markov chain M(A, v) is reversible and ergodic with stationary distribution iTg^tW 



defined in (1.2). This distribution is properly normalized. 



(2) The eigenvalues of M are {f3x}xe'Pk 5^^e^ by 



/3a 



j=i 



1 r 



Thus, (3k = l,/3fc-i,i 



q''-l 



q'^-^-l _^ q-1 



t2 ) ' 
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(3) The corresponding right eigenf unctions are 



Hp) 



h{p)=X^{q,t)l[{l-qP^] 



i=l 

with Xp{q, t) the coefficients occurring in the following expansion of the Macdonald polyno- 
mials in terms of the power sums [69\ VI (8.19)]/ 



(3.2) 



cx{q,t) 



'llil-t''^)X^{q,t) 



i=l 



Ppix)-, 



(4) The f\{p) are orthogonal in ^^(vrg^j) with 



(/a,/m) = 5x^cx{q,t)cx{q,t) 



{t,q)k 



Example 2. When k = 2, from (2.15), the matrix M with rows and columns indexed by 
2, 12, is 

t+1 t-1 



M 



1 
2t 



(g-l)(i + l) At+{q-l)it-l) 



+ 1 



q+1 



Macdonald [Ml P- 359] gives tables of K{X, p) for 2 < k < 6. For k = 2, K{\, fi) is 
( ^ ^ ) . The character matrix is ( Jj^ ]; ) , and the product is ( ^Zf t+i ) • From Theorem 
|3.l[ 3), the rows of this matrix, multiplied coordinate-wise by (1 — q"^), (1 — g)^, give the 
right eigenvectors: 

/(2)(2) = /(2)(l') = (l-9)'(l + 'Z), 
/(I2)(2) = (i-l)(l-g2)^ 

/(I2)(l2) = (t + l)(l-g)2. 

Then /(2)(p) is a constant function, and /(i2)(/9) satisfies ^ M{X, p)f(i-2^[p) = /3(x2)/(i2)(A), 
with/3(i.) = l±^. 



Further useful formulae, used in Section 5, are [391 VI Sect. 8 Ex. 8]: 
(3.3) 

Xi!'\q,t) = iq,q)kl[il-tP^) xf\q,t) = (-l)H-%)(t, i), JJ(l _ iP»)-l 



(i,i)eA 

(ij)7^(l,l) 



with the sum over standard tableaux T of shape A, and (pTiq,t) from [391 VI p. 341 (1) 
and [391 VI (7.11)]. 
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3.2 Overview of the argument 

Macdonald |39l VI] defines the Macdonald polynomials as the eigenfunctions of the operator 



D^f : A„ — )• A„ from (2.2). As described in Section 2.1 above, D^f is self-adjoint for the 
Macdonald inner product and sends A^ into itself \S9\ VI (4.15)]. For X\- k, k < n, 



iW 



(3.4) 



DltPx{x;Q,t)=PxPx{x;q,t), with ^^ = ^q^H 



k—i 



i=l 



The Markov chain M is related to an affine rescaling of the operator D^^, which |39|. VI 
(4.1)] calls En- We work directly with D^^ to give direct access to Macdonald's formulae. 
The affine rescaling is carried out at the end of |Section 3.3 below. 
The integral form of Macdonald polynomials [39, VI Sect. 8] is 

Jx{x; q, t) = cx{q, t)Px{x; q, t) 

for cx defined in (3.1). Of course, the Jx are also eigenfunctions of D^^^. The Jx may be 



expressed in terms of the shifted power sums via [i.39, VI (8.19)]: 



i{p) 



(3.5) 



Jx{x;q,t) = Y,z;'XpiQ,t)Pp{x\t), pp(x; t) = pp(x) J](l - t'^O- 



j=i 



This is our equation (3.2) above. In Proposition 3.2 below, we compute the action of D^ 



on the power sum basis: for A with £ parts, 

def 

[n]px 



'q,t 



(3.6) ^n 



t 



JC{1,...,£} keJ 



n('^'^-i) Ena-*-'^™) 



Pp 



f^^\Xj\ 



On the right, the coefficient oipxjcPfi is essentially the Markov chain M; we use M for this 
unnormalized version. Indeed, we first computed (3.6) and then recognized the operator as 
a special case of the auxiliary variables operator. 



Equations (3.4)-(3.6) show that simply scaled versions of X^ are eigenvectors of the 



matrix M as follows. From (3.4), (3.5) 



PxPx{x;q,t) = DltPx{x-q,t) = -D^^iJx) 



1 

Cx 



(3.7) 









cx 



^x^''^ 



n(i-t^o 



xm{p,i,). 
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Also, from ([slj) and ^) 

(3.8) 



PxPx{x;q,t) = ^Jx{x;q,t) = ^ J^ X^l [^(l " *"^>^( 



ex 



ex 



x). 



Equating coefficients of p^{x) on both sides of (3.7), (3.8), gives 



(3.9) 



Px 
ex 






This shows that hxip) 



x^Ud^-t^"^) 



is a left eigenfunction for M with eigenvalue Px- It 



follows from reversibility (7rq^j(p)M(p, /x) = 7rg^i(/x)M(/i, p)) that /iA(/u)/7rg,f(/u) is a right 
eigenfunction for M. Since TTq^tifJ-) = Zz~^{q,t), simple manipulations give the formulae of 
part (3) of Theorem 



3.1 



As explained in Section 2.1 above, the Macdonald polynomials diagonalize a family of 
operators D^^, < r < n. The argument above applies to all of these. In essence, the 



method consists of interpreting equations such as (3.5) as linear combinations of partitions, 
equating px with A. 

3.3 Proof of Theorem l3J] 



As in Section 2.1 above, let Dq^t{z) = Z]"=o -^g,*-^^- L^t [n] = X^"=ii"~*. The main result 
identifies Di^, operating on the power sums, as an affine transformation of the auxiliary 
variables Markov chain. The following Proposition is the first step, providing the expansion 
of D]^i acting on power sums. A related computation is in [6i, App. B Prop. 2]. 

Proposition 3.2. 

(a) If f is homogeneous, then 

Dlj = f, DlJ = q'''^^f^f, and D^^V = tdeg(/)+n(n-l)/2^deg(/)^l_^^^_^^_ 

(b) If X = (Ai, . . . , A^) is a partition then 
(3.10) Dltpx = [n]px+ 

Mh|Aj| \m=l 



E p^.i^ n 



JQ{1,...,1} 



f^m. 



Pm- 



aeJ 



Proof of Proposition 3.2. 
(a) If / is homogeneous then 

(3.11) DIJ= Y. Mx;t)\[Tq,^J = Tq,^,Tq^^,. 

/C{l,...,,i} ig/ 



rp f _ „deg(/) J- 

• • ''-q,x„j — q J 
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By definition, 



r(r-l)/2 TT ^^i ^j 



•Xj 7 Jb rj 



(3.12) Aj{x;t) = - f TT^mJ «<5 = r^^-^^/^n 

Letting x"' = xj^ ■ ■ ■ xZ" for 7 = (71, ... , 7^), 

T T T T ^7 — „7iH l-7n-7j^7i „7n 

-'i},xi-t(j,X2 • • • -'^QiXj • • • ^q,Xn-^ — y ^Y ■ ■ ■ •^n 

and it follows that 

V3--L>JJ J-q,xi'Lq^x2 ■ ■ ■ ^q,Xj ■ ■ ■ J^q,XnJ = Q ^q-^,XjJt 

if / is homogeneous. Thus, 

^.V/= E Ai{x-,t) mr.A f 

-rc{i,...,n} Vie/ / 

|/|=n-l 

n 

— 2_^ ^{j}'=(^; tjTq^xx ■ ■ ■ Tq^Xj ■ ■ ■ Tq^XnJ 

" 1 . 

— / ^ '''t,Xi ■ ■ ■ -Lt,X^ ■ ■ ■ ''-t,Xn^5^q,X\ ■ ■ ■ -'g.Xw • • • -Lq,XnJ 

^ as 

n 

n 

= t^^s(/)+"(«-i)/2,dcg(/) ^^^. (^.,-1) T^_^^^^f 

^ ^deg(/)+n(n-l)/2^deg(/)^l_^ _^ j^ 

Hence, 

(3.14) D^'-V = ideg(/)+n(n-l)/2^deg(/)^l^^^^^^ j_ 

(b) By m VI (3.7), (3.8)] 

/3eS„A \«=1 / \i=l / /3eS„A 



n(l + ^t"-0 mA = E*'^^^'"'^ 



l)/2 
=1 / r=0 



z mA, 



where m,A denotes the monomial symmetric function. Thus, since Dq^t{z) = ^r=o^qt-^^ 
and 

Dlt= E Aj{x;t)lln,,^= Yl Mx;t), 

IC{l....,n} i^I /C{l,...,n} 

\I\=r \I\^r 
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^A,{x;t)f = DlJ=[n]f 



it follows that 
(3.15) 

for a symmetric function /. By |39l VI Sect. 3 Ex. 2], 

n 

(3.16) (^ - 1) IZ ^*(^5 t)xl = egr {x- 0, r^) - 5or, 



i=l 



where, from ^39* VI (2.9)], 



9r{x;q,t) = ^zx{q,t) ^px{x), 



Xhn 



with zx{q,t) as in (1.1). 

Let A = (Ai, . . . , A^) be a partition, and 

for J C {!,...,£}, let Aj = (Aji,...,AjJ ii J = {ji, . . . ,jk}- 

Then, using 

(3.17) Tq^x.Pr = q^'xl -Xl+Pr = (/ - l)x[ +Pr, 

( |3l6l ), and ^JE\) , 



D\,tP\ = Y^ Aj{x] t)Tg^^^px, ...pxt 



n 



^ A,(x;i) ((g^^ - l) xf +pxX..[U' - l) xf +PX 



n 



i=i jc{i,...,£} VfceJ / si^j 

jc{i,...,£}s^J VfceJ / j=i 






YM^-,t)px+ Y P\jAll{l^'-^)]j^9\Xj\{x;0,t-') 



i=i 



jc{i,...,f} \keJ 



jc{i,.../} \fcej / ^hlAjl ^^ ' 



:^M 



£(^) 



npa + E ^A,. n [q'^ - 1) h^T E n(i - ^-'^'") >• ° 



jc{i,...,f} \fceJ 



Atl-|Aj| \m=l 
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Let us show that the measure TTq^t{^) is properly normahzed and compute the normal- 
ization of the eigenvectors. 



Lemma 3.3. Let 7rg,t(A) be as in (1.2) and let f\{p) = X^ Yl^^lil — q^^) he as in Theorem 
^D[3) Then 

X]^9,t(-^) = l and ^fl{p)T^q,tW = Tr-::^cxcx- 



Ahfc 



p\-k 



{t,q)k 



Proof of Lemma 3.3, From |39l VI (2. 9), (4. 9)], the Macdonald polynomial P^i^\{x;q,t) can 
be written 

{q,q)k {q,q)k 



From [Ml VI (4.11), (6.19)], 
and it follows that 



{t,q)k {t,q)k 



2^2A((?,t)-' 



Px- 



Xhk 



{Px,Px) = c'Jcx, 






X\-k 



{t,q)k 



Xhk 



iq,q)i, 



{Q,(l)kC(k) 



-A rfHp) 



To get the normalization of f\{p) = Xp ni=i(l ~ ^''0 i^ Theorem 3.1 4), use (3.5) and 



cxA = {cx?{Px,Px) = {Jx,Jx) 

( i{p) 

phk \ 



i=l 



E'-/(^.*('.')n(i-'"))'nfe|S 

p\-k i=l ^ ' 

phk ' phk 



a 



We next show that an affine renormalization of the discrete version M (3.5) of the 



Macdonald operator equals the auxiliary variables Markov chain of Section 2.3 Along with 
Macdonald [Sg VI (4.1)], define 



Ek = t-^Dlt -^t-\ and let Ek 



t 



i=l 



l'' -1 



Ek, 



operating on A^. From (3.3), the eigenvalues of E^ are /3a = YliLiil'^^ ~ 1)^ *• Noting 



that /3(;j) = ^ — , the operator E^ is a normalization of Ej. with top eigenvalue 1. From 



Proposition |3.2[ b), for A a partition with £ parts. 



jc{i,...,e}kej 



i{p) 

n 



k\ Pp 

z,,' 
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Using px as a surrogate for A as in Section 3.2 the coefficient of z^ = Ajc/x is exactly M{X, u) 
of lSection 2.31 



This completes the proof of Theorem 3.1 



n 



Example 3. When k = 2, from the definitions 

E2P2 -- 



Pl2 + — ^ — ) P2, 



E2P 



12 



2(g + 



^ ((g + 3 - gt-1 + t-l)j^i2 + (g - 1)(1 + t-l)p2) . 



Thus, on partitions of 2, the matrix of E2 is 

/ 1 + r^ 



1-t 



-1 



\ 



(l + r^)(g-l) 3 + g + t-i-t-ig 
V 2{q + l) 2(g + l) / 



This is the matrix of (2.15) derived there from the probabilistic description. 



4 Jack polynomials and Hanlon's walk 

The Jack polynomials are a one-parameter family of bases for the symmetric polynomials, 
orthogonal for the weight {px,p^)a = ct zxSxii- They are an important precursor to the 
full two-parameter Macdonald polynomial theory, containing several classical bases: the 
limits a = 0, a = 00, suitably interpreted, give the {ex},{mx} bases; a = 1 gives Schur 
functions; a = 2 gives zonal polynomials for GLn/On] a = ^ gives zonal polynomials for 
GLn{M)/Un{M) where M is the quaternions (see [39l VII]). A good deal of the combinatorial 
theory for Macdonald polynomials was first developed in the Jack case. Further, the Jack 
theory has been developed in more detail [3ll[36l[50] and [391 VI Sect. 10]. 

Hanlon [3T] managed to interpret the differential operators defining the Jack polynomials 
as the transition matrix of a Markov chai n on partitions with stationary distribution TTa{X) = 
Za~^ ' /zx, described in Section 2.4.2 above. In later work p2], this Markov chain was 
recognized as the Metropolis algorithm for generating TVa from the proposal of random 
transpositions. This gives one of the few cases where this important algorithm can be fully 
diagonalized. See [33] for a different perspective. 

Our original aim was to extend Hanlon's findings, adding a second "sufficient statistic" 
to ^(A), and discovering a Metropolis-type Markov chain with the Macdonald coefficients as 
eigenfunctions. It did not work out this way. The auxiliary variables Markov chain makes 
more vigorous moves than transpositions, and there is no Metropolis step. Nevertheless, 
as shown below, Hanlon's chain follows from interpreting a limiting case of -D^, one of 
Macdonald's V^ operators. We believe that all of the operators Dl^ should have interesting 
interpretations. In this section, we derive Hanlon's chain from the Macdonald operator 
perspective. 
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Overview 



There are several closely related operators used to develop the Jack theory. Macdonald [39] 
VI Sect. 3 Ex. 3] uses Da{u) and D''^, defined by 



(4.1) 



DJu) 



" 1 '^ / r> \ 

Y, ^>""' = -Yl det(u;)x"'^ H ( ^ + (^'^)^ + "^^Tf- ) 

r=0 '^^ wdSr. i=l^ '^ 



where b = (n — 1, n — 2, . . . , 1, 0), 05 is the Vandermonde determinant, and x^ = x^^ 
for 7 = (71, ... , 7„). He shows [39l VI Sect. 3 Ex. 3c] that 



(4.2) 



Da_{u) = lim Tr^Tv" A«,t (2;^"^) if z = (t - l)n - 1, 



so that the Jack operators are a limiting case of Macdonald polynomials. 

Macdonald [39', VI Sect. 4 Ex. 2b] shows that the Jack polynomials J" are eigenfunctions 
of Da{u) with eigenvalues /3a (a) = IHLil'" + n — i + aXi). Stanley ^50i Pf. of Th. 3.1] and 
Hanlon [3lJ (3.5)] use D{a) defined as follows. Let 



(4.3) 
(4.4) 



di 



_d_ 

dxi' 



and D[a) = aUn + Vn- 



x'i 



Jbj, Jj j 



-di, 



Hanlon computes the action of D{a) on the power sums in the form (see (4.7)) 
(4.5) D{a)px = {n-l)rpx + aij ^^^A(a)PM' 

where n is the number of variables and A is a partition of r. 

The matrix iij,\{a) can be interpreted as the transition matrix of the following Markov 
chain on the symmetric group Sr- For w G Sr, set c{w) = # cycles. If the chain is currently 
at wi, pick a transposition {i,j) uniformly; set U72 = wi{i,j). If 0(^2) = c{wi) + 1, move to 
W2. If c{w2) = c{wi) — 1, move to W2 with probability 1/a; else stay at wi. This Markov 
chain has transition matrix 

1 



Ha{wi,W2) 



Q 



1 



if W2 = wi{i,j) and c{w2) = c{wi) + 1 

if W2 = wi{i,j) and c{w2) = c{vji) — 1 
2) 

n{wi){l — a~^) 



a 



Q 



if Wi = W2 



where n{wi) = X]i(i — l)Aj for wi of cycle type A. Hanlon notes that this chain only depends 
on the conjugacy class of tui, and the induced process on conjugacy classes is still a Markov 



chain for which the transition matrix is the matrix of i^x{a) of (4.5). 



22 



The Jack polynomial theory now gives the eigenvalues of the Markov chain Ha{wi, W2), 
and shows that the corresponding eigenvectors are the coefficients when the Jack polyno- 
mials are expanded in the power sum basis. The formulae available for Jack polynomials 
then allow for a careful analysis of rates of convergence to stationarity; see [15] . 

We may see this from the present perspective as follows. 

Proposition 4.1. Let Da{u) and D{a) be defined by (4.1), (4.4). 

(a) Let D^ be the coefficient of u""* in Da{u) (see [39, VI Sect. 3 Ex. 3d]). If f is a 
homogeneous polynomial in xi, . . . ,Xn of degree r, then 

(4.6) Dlf = f, Dl{f)=(ar + ]^n{n-l)\f, and Dlf = {-a^Un - aVn + Cn)f, 



where 



-a r{r — 1) H — arn{n — 1) H -,n{n — l)(n — 2)(3n — 1). 



(b) From [501, pf. of Th. 3.1], 



2_^a\k[\k - 1) + a 2^ 



(4.7) 



D{a)px 



-^P\ 



■j+^k 



fc=l 



3,k = l 



PXjPXk 



Afe 



Afc-l 



+ Y^X^{2n-\k-l) + Y,^Y.P^ 



k=l 



fc=l^^fc m=l 



f^ — mPm 



Remark. From part (a), up to affine rescaling, D^ is the Stanley-Hanlon operator. From 
part (b), this operates on the power sums in precisely the way that the Metropolis algorithm 
operates. Indeed, multiplying a permutation w by a transposition (i, j) changes the number 
of cycles by one; the change takes place by fusing two cycles (the first term in (|4.7|)) or by 



breaking one of the cycles in w into parts (the second term in (4.7)). The final term 
constitutes the "holding" probability from the Metropolis algorithm. 



Proof of Proposition 4jJ_ 
(a) D^ is the cofficient of -u" in Da{u), so 

1 v^ 



DU 



as 



w£Sn 



det{w)x'"'f = -asf = f. 
as 



D]^ is the coefficient of u" ^ in Da{u), so 

1 " 

Dif = -y] detiw)x^^ V {iw6), + axA) f 

/ n 

= — Y, det(«;)x"'^ «^ + J^ 



^""^T^Sr+G 
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Z?Q is the coefficient of n" in Da{u), so 



(4.8) 



Dlf=^Yl I det(u;)2;'"^ J] ((w;5), + ax A) iiw5), + ax,-9j) | /. 



Since xtdiXjdj = XjdjXidi for all I < i, j < n, and XidiXjdj = XiXjdidj for i 7^ j, 



n 



^XiSjj =^Xi9iXi0i + 2 ^ XiXjdidj 

l<i<j<n 



SO that the coefficient of a in (4.8) is 



2 ; 



j=i 






y^ XidiXjdj \ f = ^ XiXjdidj f 



»(• 

1 / 



y^ XidiXidi f 



i=l 
n 



Y,X^d^+xfdf]f 



i=l 



1 2 1 1 

-r r 

2 2 2 



E-'^n/ 



i=l 



y-r)-Un]f. 



The coefficient of a*^ in equation (4.8) is 



•^ weS„ \ l<i<j<n J 

" W&Sn \ \ \i=l 



E(^^)^ - E(^'^^)' / 



j=i 



lQn2(n-l)2-ln(n-l)(2n-l))/ 
1 



24 



n(n-l)(n-2)(3n-l)/ 



since, for each w £ Sn, 

n n ^ n n ^ 



i=l 



i=l 



4 = 1 



i=l 
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Since as = E«;e5„ det(u;)x""^ = Ui<i<j<ni^i - ^i)' then, for fixed i, 
— V det( w)(wS)ix'"^ = — y" det(w)xi^^x'"^ 



05 



loeSn 



as 



weSn 



(wjx^^ = —Xidias 
as 



—Xidi y^ det( 
Is ^ — ' 

w&S„ 



.i=i 



J^ -1 J/ 2 



j=i+i 



•Xj 1 lij n 



Oi\Xi •^j) 



—Xi [as} 






ji^i 



Jb'i Jb n 



so that the coefficient of a in (4.8) is 



- y det(w;)x""^ V ((w(5)ix,a- + (it;5),x,9i) / 
Is ^-^ ^-^ 

^ Inn n 

1 1 "" 

= -n{n - l)rf - - V det(«;)x""^ y^{w6)iXidif 



-n 



(n-l)r/-^^ 



j=i j^i 



rn(n - 1) - K /. 



J/^ "^ 7 



-a^jSj/ 



25 



(b) Since dipe = ix^ ^, 

n 

1=1 

n I ^ 



Since 



then 



a 
2 



a 
2 

i=l 



1=1 



..=1 ^^^ 



P\ 



«^^2 I ^ A,A,x^-^xf--^ ^^^^ A,(A,-l) ^A,.-.^^ 



i=i \ j,fe=i 



PA.PAfc 



,=1 ^^. 



2 



a 
2 



^ ^ A^A^^x^^^ ^ ^ A,(A^-2) ^A,^^ 






,=1 P^. 



Xj{Xj-l)px+ }_^ 



i=i 



"iTi P^jP^k 



Px 



JT^k 



PX,-mPm = {^4" " \Y- 



j;x^+ J^x^-xf, 



U=l 



vi=i 



PXk—mPm PXk 



E 






™Afc-m m 
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Hence 



VnP\ = ^ 



n 9 

xf 



n 9 

xf 



Jb 1 <Xj 1 



■^^ Xj — Xn ^-^ VX. 



E 



Afe+l _ Afc+1 s . 



l<i<j<n 



—PA 



J/ 7 J/ ')' 



E 



"* -^J fc=i PAfe 



-P\ 



k=l 



P\k 



E 



_Aj. , ..Afc — 1 



Afc-1 , .Afc 



K _|_ ^ K 2; _j_ . . . _|_ 2- 2- K _j_ 2; 



l<j<j<n 



Afe 1 



EAfe J^ I v^ Afc , At-1 , , A..-1 , Afc 
PA ^ 2^ a;j '^ + j; > Xj H VxiXj + x^ 



k=iP^^ 2 






I 



^-^PA^ll^ 



Afc-1 



fc=l 



PA, ^2 

Afe 1 
— Pat, 

PAfc 2 



l)PAfc + X] (P^k-mPm - PaJ + (" - 1)PA 



m=l 



At-1 



X] — ^PA^ (2(^ - 1) - (Afc - l))PAfe + X] P^k-mPm 

fc=l^^fc \ m=l 

-. / s ''a Afc-1 \ 

2 ( X] -^^(^^ - Afc - 1)PA + X] ^^^^ X] PAfc-mPm j 
Vfc=l fc=l ^'^'= "1=1 / 



Afc-1 



k E^^(2--^^'-i) + E— E 



^fc=i 



fe=l ^^fe m=l 



P\f^—mPm 



The formula for D{a)px = {aUn + V^n)pA now follows. D 

5 Rates of convergence 

This section uses the eigenvectors and eigenvalues derived above to give rates of convergence 



for the auxiliary variables Markov chain. Section 5.1 states the main results: starting from 
the partition (k) a bounded number of steps suffice for convergence, independent of k. 



Section 5.2 contains an overview of the argument and needed lemmas. Section 5.3 gives 
the proof of Theorem 5.1 , and Section 5.4 develops the analysis starting from (1*"), showing 
that logg(A;) steps are needed. 

5.1 Statement of main results 

Fix q,t > 1 and k > 2. Let Vk be the partitions of k, vrg_((A) = Z/z\{q,t) the stationary 



distribution defined in (1.2), and M{X,u) the auxiliary variables Markov chain defined in 
Proposition 2.1 The total variation distance llAf^c-) ~^(j,t||TV used below is defined in (2.7). 

Theorem 5.1. Consider the auxiliary variables Markov chain on partitions ofk>4:. Then, 
for alie>2 



(5.1) 4 



M, 



(fc) ~ ^g,< 



< 



TV- (l-g-l)3/2(l_^-2)2 



1 1 

- + 



fqk/2 



2e 



+ k 



t-l 



k/A 
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For example, if g = 4, t = 2, and fc = 10 the bound becomes 1.76(.26)2^ + 20(1/512)2^. 
Thus, when (. = 2 the total variation distance is at most .05 in this example. 

5.2 Outline of proof and basic lemmas 



Let {/a, /3a}ai-A; be the eigenfunctions and eigenvalues of M given in Theorem 3.1 From 



Section 2?2\ for any starting state p 
(5.2) 



4||Mp - ^qAWv < 2^ --7TT = 2^ /a(p)P, 



A 



7rg,t(A) 



A^(fc) 



with /a right eigenfunctions normalized to have norm one. At the end of this subsection we 
prove the following: 

1 



(5.3) 

(5.4) 
(5.5) 



Y.fl^p) 



for any p ^Vk- 



1 



is an increasing sequence bounded by [1 — q^ )^ ' . 



(5.6) 



Px is monotone increasing in the usual partial order (moving up boxes 
in particular, (ik-i,i is the second largest eigenvalue and all /3a > 0. 

2 



A 



k—r,r 



Using these results, consider the sum on the right side of (|5.2l), for A with largest part 



Ai less than k — r. Using monotonicity, (5.5), and the bound ( |5.3[ ), 



By taking r = k/A gives the second term on the right hand side of (5.1). 
Using monotonicity again. 



(5.8) 



E fi(k)pr<j:pfUr)j:fi-r,.)(k). 






r=l 



7l-r 



The argument proceeds by looking carefully at /| and showing 



(5.9) 



fit 



f(u-r,,){k)<cf^{r 



for a constant c. In (5.9) and throughout this section, c = c{q, t) denotes a positive constant 
which depends only on q and t, but not on k. Its value may change from line to line. Using 
(5.3) on Vr shows ^^hr /a'(^) ~ '^q,t ('') ~ '^'^- Using this and (5.6) in (5.8) gives an upper 



bound 
(5.10) 



E /l(p)/3f<cE 



A/(fe) 
Ai>fc-j* 



r=l 



r. 
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This completes the outhne for starting state (k). 

This section concludes by proving the preliminary results announced above. 

Lemma 5.2. For any p G Vk, the normalized eigenf unctions f\{p) satisfy 



Xhk 



T^q,t{p)' 



Proof. The {/a} are orthonormal in L'^^iTq^t)- Fix p £ Vk and let Sp{i') = Spy be the 
measure concentrated at p. Expand the function g[v) = 5p{v)/'Kq^t{p) in this basis: g{v) = 
Y.\{9\h)f\{v)- Using the_Plancherel identity, E5(«^)%,i(i^) = Y.\{9\h)'^- Here, the left 
side equals vr"^ (p) and {g\f\) = ^^ g{v) f\{v)-Kq^t{i^) = f\{p). So the right side is the needed 
sum of squares. D 



The asymptotics in (5.4) follow from the following lemma. 



Lemma 5.3. For q,t > 1, the sequence 

P = f ^-^~' \ 1 ^ {l-q-'') q' {t,q)k ^ 'y^ 1 - t'^q^^ 
' {l-t-^J knq,tik) {l-t-^)t^{q,q)k f\ l-q-^ 

1 

is increasing and hounded by 



x/T 



7-1 



1 - t-^q-'' 



Proof. The equalities follow from the definitions of 7ry^j(A), (t, q)k and {q, q)k- Since 
> 1, the sequence is increasing. The bound follows from 

oo _ I _j I CO 

i=i ^ \i=i 

^j=l n=l 

^-p EE — ^ — 

\ n=l i=l 





\n=l 


(1- 


-r 


-n) 


q 


-n 






n 




1- 


q~ 


exp 


>: 

\n=l 


(1- 


-r 


1 







< 



exp (-llogd -,-')) = -^ 



D 
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Remark. The function P^o = liuik-^oo Pk is an analytic function of q,t for \q\,\t\ > 1, 
thoroughly studied in the classical theory of partitions [3j Sect. 2.2]. 

For the next lemma, recall the usual dominance partial order on "P^. : A > /x if Ai + • • • + 
Ai > /^i + • • • + /Xj for all i { 391 I-l]- This amounts to "moving up boxes" in the diagram 
for /i. Thus (k) is largest, (1^) smallest. When fc = 6, (5,1) > (4,2) > (3,3), but (3,3) 
and (4,1,1) are not comparable. The following result shows that the eigenvalues f3\ are 
monotone in this order. A similar monotonicity holds for the random transpositions chain 
|19j . the Ewens sampling chain |15) . and the Hecke algebra deformation chain |18j . 



Lemma 5.4. For q,t > 1, the eigenvalues 



''-?^E(^^'-i)*- 



are monotone in A. 

Proof. Consider first a partition X, i < j, with a = Xi > Xj = b, where moving one box 
from row i to row j is allowed. It must be shown that g""''^t~* + q t~^ > q"'t~^ + q t~K 
Equivalently, 



or g'^+H^"* + q^~^ > qH^~' + q^ 

or q''t^-\q-l)>q^-^{q-l). 



Since t^ * > 1 and g" *+^ > 1, this always holds. D 



q'' -I q" _ 1 



By elementary manipulations, —^ — - < —^ = ^_^ for 1 < a < 6, so that 



. , ^ t [q^-''-l g^-l\ 1 1 1 / 1 

(5.11) /?(,_,,,) = -^—-^ [—^- + -1^) -Jr+J^r=Yr\^ + J^r:^^- 



which establishes (5.6) 



5.3 Proof of Theorem 15.11 



From Theorem 3.1, the normalized eigenvectors are given by 

(5.12) U,i^ M^^^-'^) .|lk. .he.e X.^, ^ J^ (.-.-,-.) 

(ijV(l,l) 



and c\ and c'x are given by (3.1) 



Lemma 5.5. For X = {k — r, 7), with 7 h r and r < k/2, 

[i-q'')\''{t,q)kr{q,q)r 



fxiky < MrY 



{l-q-rft''{q,q)kq'-{t,q)r' 
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kj^ 



'y A 






Proof. Let A = (A; — r, 7) with 7 h r and r < k/2. Let U be the boxes in the first row of A, 
and let L be the shaded boxes in the figure above. 

For a box s in A, let i{s) be the row number and j{s) the column number of s. Then 



.^A X2 n i^,)ex {t'^'-q^-') 



'(k) 

.K), 



n (M)67 {t'-^-q'-'f 

2 



(^) _ gi('')-l 






m=l 



m=7i+l 



where 7^^ is the length of the mth column of 7. Next, 

cxc'x _ Usex (1 - g"(")t'(^)+i) (l - g«W+it'W) 
c^ ~ 11^67 (1 - g«(")t'W+i) (1 - g«W+it'W) 

set/ 

71 k—r 



k—r—m. 



t 1 



k—r— 171+ 



m=l 



m=7i + l 



-2(fc-r)(fc-r-l) 



71 



Thus, 



m=l 



Yl f tT;.+igfc-r-i _ qni-i\ IrLq 



^k—r m—1 



k—r 



m=7i+l 



' X^ \ ^ ^ ^' 71 



,fe— r „m—l 



x^ 






(tT™ - g™-l) 



(i' 



7„ 



„m— 1"| 



W ] ^7S ^ 2(fc-r)(fc-r-l) TT 



k—r 



:i 



^m— In 



(l-(7 



m— 1"! 



-I- -I- {tq^-"--^ - g™-i) (g'^-'' - q^-^) 



m,=7i+l 



■) 
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Since /c - r - 1 > m - 1 and t > 1, then tTm+ig^-^-i _ g^-i > o, so that q-^'^-'^-^h-^ < 1 
imphes 



< 



-(fe-r-l).-l 



Similarly, since A; — r > m, — 1 and t > 1, then f^^q — q^~^ > 0, so that q~^^~'^> < 1 
implies 

Similarly, t~^q~^^~^'~'^' and q'^^""^' < 1 imply 



< 



-(fc-r) 



[tq^-'--'^ - g™-i; 



< 



t-iQ-('=-''"i) and 



(gfc-r _qm-l) 



< 



-{k-r) 



So 



'X/ 



M I '^ '•y ^ „2(fc-r)(fc 



.^(oy ^^^A 



Thus, 



71 

•-' n (^ 



k—r 



-{k-r-l).-l\ L,-(k-T) 



i"^ u 



n ^-^ 



(fc-r-l)\ L,-(k^r) 



m=l 



m=7i+l 



^2(fc-r)(fc-r-l)^-(fc-r)^-(fc-r)2^-{fc-r-l){fc-r) ^ ^_(fc-r)^-(fc-r)_ 



■' {^k _ -,\2 



hiky _ / ^ W \ c^c:^ ('Z'^ - 1)' it, q)k (g, q) 



f,{TY \Xl^j cxc'^{qr-iy{q,q),{t,i 
1 {q''-l?{t,q)kiq,q)r 



< 



f-'t^-' {q' -^Y {q,q)k {t,q\ 



D 



We may now bound the upper bound sum on the right hand side of (5.2 ). Fix j* < k/2. 
Using monotonicity (Lemma 5.4), Lemma 5.2, Lemma 5.3, and the definition of 7rq^t(r) from 



(L2) 



r=l X={k-r,"f) r=l A=(fc-r,7) 



Ai>fc-i* 



^\^p2i V- F . .2 (1-9 ^Y q'" {t,q)k f {q,q)r 

- 1^ Pik-r,r) 2^ Mr) _ ^, ^^ 






< 



^^2i 1 (i-r')' g' (t,g)fc ^Mg,g) 

^ ^("-^'^'^ TTgAr) (1 - g-0' i'^ (q, q)k q' it, q\ 






< 



< 



^--)tiis 



r/3: 



2£ 



(fc-r,r)(^_^-r-)(i_^-r-) 
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Using (5.11) and Lemma 5.3 gives 
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by Lemma 5.2 Choose j* (of order A;/4) so that k — 2j* + 1 = k/2. Then 
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This completes the proof of Theorem 5.1 
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5.4 Bounds starting at (l'^) 

We have not worked as seriously at bounding the chain starting from the partition (1 ). The 
following results show that log„(/c) steps are required, and offer evidence for the conjecture 
that logg{k) + 9 steps suffice (where the distance to stationarity tends to zero with 6, so 
there is a sharp cutoff at log^(/c)). 



The L^ or chi-square distance on the right hand side of (5.2) has first term /3|1]^ i/|-i li^^) 
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Lemma 5.6. For fixed q,t > 1, as k tends to infinity, 

2 



k\2 



/(fc-l,l)(l ) 






^>^-^- 



Proof. From (3.3) and the definition of (^^(g, t) from [39, VI p. 341 (1)] and [Ml VI (7.11)], 
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from (1.2) 



Using the definition of C(fc_i i) and c',^-^ -|^n from (3.1 ), and the definition of (t, q)k and ((7, g)^ 



/( 
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and the result follows, since p ^ k for A; large. 
Corollary 5.7. There is a constant c such that, for all k,i >2, 
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2 ,,,_v^ (M^((l^),A)-^,,t(A))- ^ fl-q 

X(ife)W - 2^ — ^ 
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1^ fc^ 
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Proof. Using only the lead term in the expression for X(ik\{-(^) in (5.4) gives the lower bound 



Pfk-i,i)f{k-i,i)('^'')- '^^^ formula for /3(fc_i,i) in Theorem 3.1^2) gives /3(fc_i,i) > ^, and the 



result then follows from Lemma m£j 
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The corollary shows that if £ = logg{k) + 6, xlki^) ^ -^- Thus, more than logg(/c) steps 
are required to drive the chi-square distance to zero. In many examples, the asymptotics of 



the lead term in the bound (5.2) sharply controls the behavior of total variation and chi- 



square convergence. We conjecture this is the case here, and that there is a sharp cut-off 
at logg(fe). 

It is easy to give a total variation lower bound: 
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Proposition 5.8. For the auxiliary variables chain M{X, A'), after i steps with ' 
9, for k large and < — ^Ei? 



log„(A;)+ 



M(V) 



vr, 



q,t 



TV 



> e 



+ o(l) 



Proof. Consider the Markov chain starting from A = (l'^). At each stage, the algorithm 



chooses some parts of the current partition to discard, with probabihty given by (1.3) 



Prom the detailed description given in Section 2.4.3 the chance of a specific singleton being 



eliminated is l/g. Of course, in the replacement stage (1.4) this (and more singletons) may 
reappear. Let T be the first time that all of the original singletons have been removed at 
least once; this T depends on the history of the entire Markov chain. Then T is distributed as 
the maximum of k independent geometric random variables {XjjJL-j^ with P{Xi > £) = 1/q 
(here Xi is the first time that the ith singleton is removed). 
Let A = {\ £ Vk ■■ ai(A) > 0}. From the definition 
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From the limiting results in Section 2.4.5 under vTq^t, ai(A) has an approximate Poisson 



distribution. Thus, iiq^ti-^) ~ 1 — e '"^ • The result follows. 
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